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Thirty-six second^ semester fres.hmen' students on 
academic probation wer^ m'atched according to sex, high school class 
rankr S.A.T. scores, and. IQ. Ihey were divided into'two groups: 
control and experimental. Both groups were administered the Survey of 
Study Habits and Attitudes, after which the experimental group 
, participated in foup one-hour study skills sessions. The members of 
the gontroj group, diji not participate in any program. After the last 
study skills session both groups were again administered the Survey 
of Study Habits and Attitudes. When the differences in secures of the 
pte-tests and post-t;est:?^ of both groups were compared, there was a 
significant improvement in the scores of the experimental group. 
(Author) 



* Documents acquired by ERIC include many informal unpublished * 

* materials i.ot availa^ble Irom other sources. ERIC makes every effort * 

* to obtain the best copy available. Nevertheless, itejns of marginal * 

* reproducibility ate often encountered and this affects the quality * 

* of the microfiche and hardcopy reproductions ERIC makes available * 

* via the ERIC Document Reproductioii Service (EDRS) . EDES is not * 

* responsible for the quality of the original document. Reproductions * 

* supplied by EDRS are the best that can be made from the original. * 



ERLC 



AX 



\ 



CENTER FOR EDu6AT»0NAL>bEVEL0^MENT 
UNIVERSITY OF MINNESOTA >/ ' ! ' 

monogrdph 



CD < 
UJ 



A traiilmg system 
for graduate student 
instructors 
of economies 



' u oil 



ERIC 



number one 
fall, 1975 



•3- 



received special tutoring, and were housed together, achieved above the 
predicted grade point avera^^e. No significant difference appeared in 
the grade point average of the remaining two j'.roups which were housec 
with other freshmen, or on campus r<andomly. 

In a study conducted by Caple, (19^>9) two groups of second semester 
freshmen who received less than a 2.0 average in a ^.0 system were 
matched on the composite score of the American College Test, age, and 
academir: load. One group attended a study session which lasted for two 
hoars in duration oach evening. The program ran four evenings a week 
for a period of ei(^ht weeks. At the end of the semester, there was no 
significant difference in grade point average between the two groups. 

This study was repeated in the fall of the next academic year in- 
cludini?: a third group which underwent not only the study sessions, but 
also received tutorial help. Tn this study no significant difference 
app'^ared in the grade point avera^^^e of any of the groups. Caple then 
concluded that organized study sessions have no significant effect upon 
th» academic achievement' />f students- 

From^Ed.D., (1971) conducted a study to evalua^ the effects of 
peer tutorin?^ and individual and group counseling, with and without 
reinforcement, on the academic achievement of high rllsk students. He 
reported that those students who received tutoring and reinforced indi- 
vidual counseling earned higher weekly quiz grades than those students 
who did not receive this treatment. The students' final grades were, 
however, not significantly affected by these sessions. 

Some inctitubions have offered study skills seminars or classes 

without first conducting action research or pilot studies. This approach 
\ 
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1. Title 

The roseirch praoticimi is ontitiod "The EtT<-clu of A Study Skills 
Workshop on Second Semestor Freshmen on Academic Probation". 

2. Statement of the Problem 

Thp problem in this praoticum poses the question "Is it 'possible to 
Improve the study skills of second semester Freshmen on Academic Pro- 
bation Ihrourh their attendance at four one-hour study skills seminars?" 
3- Hypothesis 

The hypothesis states that there will be no significant difference 
in the pr^-test, ]^st-test results of second semester Freshmen Students 
on Academic Probation -elative to improvement in their study ckills, 
wh^n ^omjarei to a matched control group. 

Ba^k<^round and Sirni ficance of the Study 

Colleges and universities across the nation are face<l with several 
problems yi today's society. One of the most pressing problems is that 
of de?reasin;^ enrollment. This problem has arisen for several reasons. 
In toiay's society there is a smaller oollejPje-ace population to draw 
from. ?-!ore individuals are' questioning the importance of a college 
^education when they see the larj^e niijnber of collep^e graduates unem- 
ployed, or omployM in some^ capacity other than that for which they 
have been trained. Many coUef^e graduates are earning less money than 
non-colle^^e ;^radaate.<?, Berendzen, (I968) relates a study completed by 
the Offi'-e of Higher Education for New York State. He outlines three 
i<ientifyin;j factors contribuf inr: to this decline in enrollment; (a) the 
attitude chani-^e in high school graduates concerning the need for a 
college education, (b) a rr^-at amount of publicity concerning the 
limit^^l ,?ob opportunities of college graduates, (c) changes in the 




military draft law. . Tbe problem of enrollment is faced even more 
dramatically by private irrstitutions, where -^ublic support is virtually 
non-existent, and costs are constantly rising. 

Data collected during a * survey by" William Jellema, (1971) further 
supports the fact that enrollment- has been decreasing in private in- 
stitutions. Jellema states that decreasing enrollment affects not only 
institutional size but financial stability as well. In order to sur- 
vive, they mu3t incorporate in their regular colleee functions bett\pr 
studies and attractive academic programs. 

Several institutions, both public and private, have begun to re- 
search the problem of attrition. The significance pf this study deals 
not so much with the overall subject of attritic^. It does, however, 
su^'gest that a contributing; factor in attrition is the number of students 
who leave collef^e because they are not able to achieve the academic 
standards of their institutions. Many also leave because they fail to 
meet their own goals for achievf^ment . Cope, (I968) relating a study 
conducted by Walker, attributed one-third of college withdrawals for 
academic reasons, and one-third to motivational forces. The remaining 
one-third was attributed to financial difficulties. 

Elizabeth Shafer Ph.D., (I969) conducted a study at^Madison College 
to determine if a residence-hall student tutorial-counseling program, 
combined with a systf^matic method of room assignment according to 
ability, would have an effect on low ability freshmen^omen. Sh?fer 
ucei ^hr^e f:roups of freshmen women. Only members of 'I'he group which 
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received special tutoring, and were housed together, achieved above the 
predicted grade point averar,e. No significant difference appeared in 
the grade point average of the remaining two j'.roups which were house 
with other freshmen, or on campus r.andomly. 

In a study conducted by Caple, (19^9) two groups of second semester 
freshmen who received less than a 2.0 average in a ^.0 system were 
matched on the composite score of the American College Test, age, and 
academi*^ load. One group attended a study session which lasted for two 
hours in duration f^ach evening. The program ran four evenings a week 
for a period of eif.ht weeks. At the end of the semester, tliere was no 
significant difference in grade point average between the two groups. 

This study was repeated in the fall of the next academic year in- 
cluding a third group which underwent not only the study sessions, but 
also received tutorial help. Tn this study no sip;nificant diffe/once 
appeared in the grade point averaf^e of any of the groups. Caple then 
concluded that organized study sessions have no sip;nificant effect upon 
th«=* academic achievement" />f students. 

From^Ed.D., (1971) conducted a study to evaluat^ the effects of 
peer tutoring and individual and group counseling, with and without 
reinforcement, on the academic achievement of high rilsk students. He 
reported that those students who received tutoring and reinforced indi- 
vidual counseling earned higher weekly quiz grades than those students 
who did not receive this treatment. The students* final grades were, 
however, not significantly affected by these sessions. 

Some institutions have offered study skills seminars or classes 

without first conducting action research or pilot studies. This approach 
\ 
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was us»>'i al the University of Misr.issippi . Wilson, (190^5) explains this 

stuiy. Two ^^roups of Liberal Arts students wero matched, accorJinn to 

'the Composite Standari soore on the American CoU ^re Test, matriculation 

date, a:e and sr^x* One ^'iroup attended the study sklLLs clans, owi 
4m 

groa^ did not. Wilson reported that there was no significant difference 
in academic achievement between either croups. ^o^^^9ii^:^^Ji^g^f:ir ^ did 
achieve an increase in men r.rade point average. 

3yk*»s, ("'971) reported his study as one whereby five groups of ) 
S'Uient,^ wex-i treated in viriois manners to determine possible causes of 
attrition. Ho utili'*:e<] tut.orinf^, reading Instruction, and financial 
assistance to measure thei r Njf feots on attrition and firade point 
averages. This study did nojL indicate that the c^ade point avera^^o 
for student's in any of the Groups chanj^ed significantly. It was noted 
that the readim^ skills of som^-* of the s^^udents did improve, but there 
wa3 not f^nou^-^h eviience to in iioate si?^nificant differences. 

Tn con^^lusion, it seems essenMal to rccof^nize that institutions 
feel the importance of such investigation as aforementioned. If grade 
point avera -e does affect attrition, and decreasing enrollments sgem to 
Y'-' a tr^nd, for private institutions in particular, it seems si^^nificant 

^ hat '^ork ^olle^^je shcild also undertake a similar study. It is, however, 

/ 

/ 

this authors opinion that administrators must not only consider the ^ 
iollars ani ?ents aspect of attrition. There also exists a moral respon-> 
silility of the institution to admit only those students who can achieve 
aoai^mic* su-^'^^ss. If students are not xchievin/ academic success, insti- 
tutions must investi^'ate the problem. • 
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Table I deals with the calculations performed on the differences 
in the pre^test, post-test s/jores of the control group. It records 
« the number, sua, sum squared, standaird deviation, mean, and variance 
^* respectively. ^ ^ 

Table II deal^^with the calculations performed on the differences 
in the pre-test, post-test scores of the experimental group. It also 
records the numbei?, sum, sum squared, standard deviation, mean and 

• * ' . ■ . 

-variance respectively. * , 

Table III records the critical, and calculated -t- values respectively. 

^ The null hypothesis cgun be rejected. The' differences in the scores 

Of the pre-test, P9strtest results of those students urtd^rgoing a study 

sfcij.13 program', when compared vO the differences in the scores of those 

students who did not, were significantly different. 
• ^ * • 

^11. Conclusions and Significance 
' * : — 

, • -a.^ The data clearly i^Q^sJicates that there is a significant difference 

* • » 

in the pre«*test, post-test/ results of second semester freshman 
s€udelii|i on academic prbbation relative to improvement in their 
9 tuj^S^*^ skills after p?^rticlpating in 'fpuiEi/cyie-hour study skills 
seminar^. ^ \ ^ ^ r 

b. .Implications fpr TOr • CQ?.l^ge, ' t 

■ *: ^' 

1) The results of this St'idy Vill lead the college to conduct 
^ * - . further r^earc^.^fl^'ttiis area. 

- M. . r 

vf-M 2) s^ther. study ^ki lis .seminar programs may be established io 
^/V ' „ • ' ' r ^ 

* ^^^^der to serve a dual purpose - to lower the attrition rate 
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' utilizing lecture/discussion and- techniques of practical- appUca- 
tion focusing on ^ structured system of study; Survey, Question, 
Read, Recite and Review, (S.Q, 3R)« 
j. Survey of Study Habits and Attitudes - Survey developed by Brown 
and Holtzman. Copyright 1967 by The Psychological Corporation, 
Wew York, New York, for the purpose of measuring study habits and 
attitudes. 

k, S.Q.3R. - System of studjr; survey, qtiestion, read, recite and 
review. 

1. ^S.A.T. - Scholastic Aptitude Test, 
T,ifnl tat ions of the Study 

The following are understood, to be recognized limitations of the 

a- The sawiple was Jra^n from a restricted population - only those 
second semester freshmen on academic probation. 

b. The study focused on a relatively small sample - 3^^ students. 

cJ The effectiveness of the scale used as- a -measure in the study 
should most certainly be recognized as a possible llmitjition. 

i. The intervening; variables and basic assumptions will also in- 
fluence the validity and accurapy of the study. ^ 

BasiCy Assumptions of the Study 

a. It is assumed that because the 3*udents selected for the study 
are on academic probation that there is a problem with their 
study skills. ^ 

b. It is assumed that the study skills which was presented (S.Q.3R-) 
.focused upon the problem which* these students are encountering. 



q. It is assumed that the students involved in this study attended 

all four sessions of the study skills workshop. 

•* 

d. It is further assumed that 4ho students were sincerely interested 
in each of zhe workshop sessions and that tl>eir attitudes reflected 
their recognition of the imT)ortance of the sessions.- 

e. It is assumed that the presentations which the senior education 

I majors made were consistent with the standards of good teaching. 

[ 

f. Tt Is also assumed that the physical facilities where the work- 
shop was conducted assisted to provide for maximtim learrfing- to * 
Qccur. ^ ' J 

^% U is assumed, that the limitations of this study as*previously 

* 

stated did not adversely aff*ect the results, 
n. It is further assumod that the stuaents accurately reported 

their responses on the pre-test, post-test, 
i. It is finally assumed that this study was of significant importance 

to warrant the time and effort expended in\arrying it out. ' 
Procedure for Collecting the Data, 

The study utilized two j^routfeof second, semester resident fresh- 

man students on aa^demic prQ^bati)on. . " ^ 

b. The Colle;fe records office and Housing office provided the names 
of the students who will fit into the a^pve mentioned groups. 

c. rortact was made with th^ York College of Pennsylvania Education 

Department to secure the names of senior education majors who 

would be interested in partaking in the study. Four of the 

« 

recommended students were selected and ma^ie the study skills • 
presentations. Each student taught one session eacb and re- 

*10 



ceivei a stipond of ton dollars^ each, 

d. The two grfups of fneshman students were patched aceordinr; %o 
sex, high school class rank, S.A.T, scores, and I,Q. ' ^ 

e. After matchini, they were labeled a contrc|t proup and an experi- 
mental group r<^spectivcly, ' - 

ft Letters were sent to each freshman selected for the study. 

• These letters explained the study, emphasized its importantje 

to them and the college, and invited thciif to take part -in the 

A- 

pro ram or parts of,the program \pre-tcst, post-test for the 

f 

matched experimental c^oup) - sample letters included in 
appendix. 

Dormitory supervisors aid fl^r advisors were utilized for 
follow-up to the letters. ^ 

h. The pre-tect and post-test ^for each group was the Surveyof 
Study Habits and Attitudes (196?). 

i. After the results of the pre-test were obtained, the four 
one-hour sessions of the workshpp ,be*^an for the experimental 
^roup. After the isist session both groups were again assembled 
for ^he post-tost evaluation. 

j. Each of the senior education majors who made presentations 
' had bf^en assigned material which was to be covered. They also 
Gubnitte i lesson plans and Wesson objectives foi* approval be- . 
fore each session. 
Procedure for Treatment of the Data i . ^ 

'"he .^lurvey of Study Habits and Attitudes was hand scored in the 
ft 

n 



mapner sugj^ested in the I907 .>urvo/ manual. Thf» pre-LesL and 
post-test results -of both s.-roups' are compared in the following 
manner. * , 

a. Number =36 , " 

b. Degrees of freedom = 30 

. 0. Nuir hypothesis = Hq : X = X2 
i. Alternate hypothesis = : Xi = X2 
e.. Level of significinco = .05 
r. rriticil -t- value « 1.697 (one .tailed test) 

Ho will be reflected and accepted if t> 1 .*bV7. 
Data Res^lMn^: from Th*- Study '1 ^ , 

The followin,- 1^.; t lie daLa rosultinr. from thg ;!tudy. 

Tublo I ' 

Control Group ' . .■ 

... HA = 16 

«.» =21 

^ , = c.-i>\ 

T-tbie TI * , 
Experiment il QroUp 

f^y = 18 

K = 2.iO - 

. <r«'- = 5.6t ' ♦ 

fable III 

Critical -t- Valup , Caictilat.e^ -t- Value 

'-••^^7 -sTTTTB 
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Table I deals with the calculations performed on the differences 
In the pre*test| post-test g^cores of the control group. It records 
• the number, sum, sum squared, standard deviation, mean, and variance 
«^ respectively. * 

Tible II dealg^with the calculations performed on the differences 
in the pre -test, post-test scores of the experimental group. It also 
reconds the numbej^, sum, sum squared, standard deviation, mean and 

• * ' . • . 

variance respectively. - 

Table III records the critical, and calculated -t- values respectively. 

The null hypothesis cgin be rejected. The' differences in the scores 

of the pre-test, post.-test results of those students urtdirgoing a^study 

sfcij.ls program-, when compared vO the di Inferences in the scores of those 

students who did not, were significantly different. 

11 • Conclusions and Significance 
— 1 a : 

♦ -a.^ The data clearly i^jjicates that there is a significant difference 
in Xhe pre-test, post-test/ results of second semester freshman 
sfudentii on academic t)rbbation relative to improvement in their 
^tuj^y skills after p?^rticipating in 'fpuij/qne-hour study skills 

seminars. ^ i ^ • ^ * 

J' 1 

b. implications fpr YBr - Co^Ll^ge. ' I 

I) The results of this stidy vill lead the college to conduct 



. ^ further riseardii^/i'flf'this area. 



2) s^ther.. study aki lis .seminar programs may be established io 
•S^* :;^^der to serve a dual purpose - to lower the attrition rate 

^ • ■ f > ' ■ 



ERJC 



cell, or, in the f ase of a multiple choice test, by the number of^ 
quijslioos which fall into eacb cell Jhese weights are represented* 
m Table j as the aij's ' 

For 'evaluating their own teaching effectiveness, as opposed to 
eyaluating student achievement, GSI s are encouraged to use 
multiple choice tests. In contrast with matching, essay, or true- 
false teStSj, multiple choice tests have the advantages of structur- 
ing responses and being' most amenable to diagnosis. To obtain 
student feedback on teaching via multiple choice test item re- 
sponses, the instructor using ^elf-made classroom tests can use a ' 
variant of discriminant analysis. 'Starfdarljized or cumulative test 
score norms'can also dsed to assess teaching^ effectiveness 

In the case of selj-made classroom tests, individual test ques- 
tion discrimiaant^ analysis is perfornr|ed by computing* a number 
which IS called the test iterri discrimination coefficient, d. This 
computation can be dpne by subtractmg, thQ number of students ^ 
who received total test scores m the. lowest third, L, but got the 
.Item rtght, r^, from the number oj students in the upper third, U, 
' who also got. this item right, r^. This dtfferen.ce is then divided, by 
one-half the number of Students m bothr groups, V2,(L+U), i.^., 

- . Vz (U + L) 

■ * ' 

•lii^aximum discriminating power would of course occur where 
d = 1.0.<niis indicates that the test item perfectly differentiated, 
students jrha^ordance with their overall achievement. A zero in- 
dicates no dis&Timination. Individual test item discrimination co- 
efficients are typhcally used to decide which^questtons in a test 
are good segregators Of students (Isaacand fvlichael 1971, 
50). "However, they can also provide the instructor with a basis on 
which to assess teaching effectiveness 4n terms of content cover-^ 
age and student ability to apply concepts learned. / ■ 

After computing a discrimination coefficient for Q^h test 
question, the instructor can gam infofmation pn his or herisach- 
ing effectiveness by computing the average discnmination power 
of all questions m a cell— that IS, the averag^e^^^discVimmation co- 
efficient for^ all questions in the'^dth applicatioir^[rr^-41h,c^^ 
cat.egory of the Table of Specification. Similarly, the msfrtr^r^ 
can compute the average discnmirration coefficient for all ques- 
tions in the conterU columns of the T^^le of Specification— that is. 
the average discriminatiorv coefficient , for all questions in the jth 
cbntent category, ' - " 

Although the discrimination power of any given test question 
^' 'hng into one of fthe Table of Specification cells may be pOo?. it. 



among York College students, and to further develop the 
skills and attitudes of York College students in a most 
positive way. 
c. Implications for other institutions. 

l) "'Being aware of the limitations of this study, along with 
the varying Vesults which have been published or reported 

4f «> * 

from other institutions regarding similar studies, it should 
be understood that siprnificant results may occur. It is 

« » 

the opinion of this author, however, that other variables 
not mentioned in this, or other studies may have an effect 
on the results Some of these variables would be the 
/ ' maturation and motivatioa factors ef fee tin;, students. 

Limitations of the study, 

1) the sample was drawn froin a restricted population only 
those second semester freshman on academic probation, 

2) It focused on a relatively small sample - 3(> st-udents. 

3) The effectiveness of the scale being utilized as the 
measure in this study, 

h) The inters/ening variables such as individual values and 
maturation factor alonf; with the basic assumptions may 
also effect the validity of the study, 
12. Further Studies 

After discussion with officials in the student affairs division of 
the college it Is apparent that the results of this study are slgnif- 
icant enou^'h to warrant further research in this area. The first 



. follovr-up to be considered will compare the first and second semoater 
..grade point averages of the students in the, experimental group. 
Though the pfe-test. post-test score results were significantly diff- 
erent for thi^s. group, application of those skills apparently learned 

'*■ I 

through the program is equally important to know. The' grade point 
averages Vould be compared again to those averages of the students 
In the control group. 

It would be interesting to study students in their second and 
third year who have fallen into similar academic situations. They 
may be maintaining a grade point average just high enough to remain 
in school, but far from their possible potential. Is it possible 
that definite study habits and attitudes might be so ingrained in 
the individual by his second or third year in college that they would 
be nearly impossible to change? 

, There are several .other studies, too numerous to mention, 
wMch would be warranted as either follow up to this study, or 
similar to this study., it is the hope of the stud.'nt affairs- division 
to engdge in several of the..e research projects in the next three years 
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YORK COLLEGE OF PENNSYLVANIA 

r 

(Experimental Group) 

February 



jr 



Dear • • v 

The- Student Affairs division of York College is pres.ently undertaking , 
a pilot study to determine the effects of^ study skills seminaurs on 
freshman resident students. You haye been selected to participate in 
thia- iii5>ortant research ttudy. It will assist you to evaluate youx'^ 
own study skills as well as present some new ideas for your consider- 
ation • The results of this study will be not only of importance to 
you but also to our college. 

Fdur short study skills seminars will be held on.Tuesdky and Thursday % 
afternoons during the weeks of February 2k and March 3, The pre- 
evaluation will be held on Thursday, February 20, 1975, Th» post^ . 
evaluation will be held on Tuesday, March 11, 1975. All sessions wUL , 
be held in the conference room of the Administration Building at 
^:00 P.M. If you will not b^ able to attend, please notify my office 
at the earliest possible convenience. 

Thank you for your titne and effort in this endeavor. 

Sincerely, 

John J. Pierog ' 

Director of Financial Aid 

Student Affairs Office 



• o . ' ... 

L: eric , . ■ • . 



YORK COLLEGE OF PENNSYLVANIA 

€Oi'i«nv tat iOAO. vmt. MNNimAi^ a im% 

(Control Group) 

' I 

February 



Dear 

You are being asked to participate in two study skills surveys. 
These surveys will be held at kiOO P.M. in the conference room of 
the Administration Buiflding on Thursday, February 20, 1975 and 
Tuesday, March 11, 1975. They will take only a short time to 
complete and the results will be of great assistance to your 
fellow students as well as the Student Affairs Division of the 
college. ' 

Thank you for your time and effort in this endeavor. 

Sincerely, 



John J. Pierog 

Director of FinancialAid 

Student Affairs Office 
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equipment simpte to operate and easy to set up and take down. 
After an hour of instruction^nd a few supervised Japing§ any re- 
' sponsible university studentVs capable of, taping an instructor in 
the classroom To get consisfient results from the taping, however. 
\\ 1$ n(3t sufficient that the t^er kno^w only how to operate the 
Equipment She or he must al^ follow a taping procedure check- 
liBt. A checklist of things tp doVwhile taping typically includes the 



lllowing' 



\ 



i(1) Set up equipment befo^re ^ class starts, ^ 

(2) Make sure equjpnni>Ql JS ooeratmg properly, 

(3) Videotape everything written on the chalk board or over- 
head projectotijiic/asec/p; \ 

(4) Try to pick up aaylITTrrgHax3ij|u^l in the classroom, 

(5) Get the students on camera vvh^Trlhey speak, and 

(6) Follow the instructor at all oyier times. , 



At Minnesota an undergraduate cfees a)l the actual taping whife 
a junior faculty member does thp *?^viewing and critiquing. The 
issue of whether the reviewer/critiqu( r should also do the taping is 
an open one. A reviewof'/critiguer whT 
'the class first hand On the other hanVi, the taping quality is some- 
times better if the taper is solel> concerned With following the 
outhned procedure 



The basic problem involved in arrAiging for classroom taping 
IS one of coordir^ation. T(je GSI mustwe notified in advance in 
order to be able to explain to the classVvhat will transpire on the 
day of taping. If the students are not irVfdrmed in advance, staff 
experience has shoyyn that the presence \\ the camera and equip- 
ment will disturb them This is true even\hough the video equip- 
ment )s quiet and requires no extra llg|iting 

Picking a time to tape the class is riot always easy. The chosen 
time must bd^ satisfactory to both the person doing^^the taping and 
the instructor. Th$. reviewer/critiquer should have some time avail- 
able shprtly thereafter, it is usually be^t to review and critique as 
Soon after the taping as possible It is a so necessary to mak^^ure 
that the class activity js appropriate— ;e g , an exam time is not 
appropriate. Finally, before taping be )ins the instructo; should 
have filled ouPthe Daily Teaching Chep list shown in Table 6. This 
fdjrm simply requires the tSSI to state, n an objective-setting ap- 
prbach, what will be ta^jght and how il^iyilLbe taught on the day of 
taping As will be shown, this form pro};ides valuable information 
for the reviewer/cntiquer'as well as forjlhe instructor being taped. 

It IS worth noting that th6 reviewer/critiquer can reduce co- 
ordination problems -by .attending the weekly seminars. Since aH 
,G§rs are required {to participate in 
schedules can be ajrranged p(ior toi 
^_.ons 



he seminars, videotaping 
or just after the seminar 
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Reviewing classroom perforrrt'ance » 

Reviewing consists of three separate procedures, coding, fill- 
ing out the Summary Checklist, and preparing the critique 

Coding The coding process requires the specially developed 
Videotape, Review ^'ding form shown in Table 7 and a (jevice' 
which will deliver beeps at 20-tfecond intervals. On hearing the 
beep, the reviewer/critiquer records qn the coding form wh^s 
going on at that moment ih the classroom. After doing this for^f^ 
entire tape, one has a time series index of what methods were 
used, what type of learning was takir^g place, and what verbal and 
non-verbal expressions the instructor projected. This information' 
serves two purposes First, it is valuable infbrmation for the re^ 
viewer/critiquer to have fof subsequent discussion. Second, it< 
'f>rov ides the instructor with a measure of wh^t he or she did tn* 
the classroom to cohipare with what he or she planned, as r^i^ 
corded on the olily Teaching Checklist which the instructor filled . 
out before the taping For example, an instructor who planned to 
lecture 25 pqrcent of the'time might not realize he of she actually 
lectured 75 percent of the time, until convinced by the coding 
data. 

Ti)e reviewer's coding forirt currently us6d at Minnesota is an 
observation instrument specially adapted for this project. It is used' 
by the reviewer/critiquer to.get a judgmental measure of (a) the 
method employed by the instructor (discussion, questions/prob- 
lems: lecture, other), (b) the learning okJfectives (complex applica- 
tion, simple application, exposition on theory, theoretical con- 
cepis. knowledge of facts), and (c) andT (d) the verbal ^nd non-* 
verbal expressions (supportive, receptive, neutral, unreceptive, or 
(Jisapprovi/^g), , 




The following operational definitions are used in coding. 
^) me[\)o6 . ' ^. 

Discussion — If during the mterval of observation the 
critiquer observes tiae teacher primarily listening to a 
student-initiated point or qi/estion and briefly respond-/ 
ing to It, It is'recorded as discussion (4). 
QuestionslProblems — If during the interval of observa- 
tron- the critiquer observes the teacher primarily asking 
questions to which he or she expects an immediate stu- 
dent response, it is recorded as questioning (3). " 
Lecture — If during the interval of observation the'criti-. 
quer observes the teacher primarily "talking at" the stu- 
dent, it IS recorded as lecturing (2). 
Other.— U tluring the observation period the j^r^tiquer 
observes the teacher doing such things as reading di- 
rectly out of the texf or watching a film with the class, 
the critiquer records "other" (1). 



/ 

/' 



ERIC 



b) Learning Objectives 

Complex application— The discussion, lecture, or qu^s-, 
tioning observed pertains td^ utilizing more than one 
economics principle or concept in analyzing a real ^ 
^ * world problem (5). 

Simple application— Jhe discussion, lecture, or ques- 
tioning observed pertains to using a single economics 
principle or concept in analyzing real world or hypo- 
thetical problems (4). 
V , Exposition on tfieory—JHe discussion, lecture, or ques- 

tioning observed pertains to proofs, intuitive explana- 
j tions, or analysis, of economic theory (3). , 

Theoretical concepts — The discussion, lecture, or ques- 
.*» tioning observed pertains to introducing and defining 
economic concepts such as comlparative advantage, 
opportunity cost, or law of demand (2). ' . / 
Knowledge of fads — The discussiorl, lecture^ or que^r*" 
^ tioning observed on'iy pertains to institutionral descriji- 

* tion^, dates, names, and the like (1), / 

c) Verbal Explessions ^ ' / 

SupportivelReceptive—\i in the process^ of lecturmg,> 
discussing, or questioning the mstructoryKeeps students 
on task or reinforces student activity by positive verbal 
comments or by changing tone of voi<^, speed of talk- 
ing, or diction, the inslwctor iS recorde^'d as being re- ^ 
ceptive. For ^example, while lectu/ing the instructor 
slows down to giv^ students ampl^ opportunity to take 
specific notes, while questioning students the instruc- 
«. tor, through probing techniques, leads the students to 
the correct answer and then congratulates them for 
critical thinking. Receptive (5) or highly receptive (4) is 
used to indicate the degree of support expressed, ^ 
20 ' ^ Neutral'— The instructor is not observed to* be changmg . 
/ \ ' .speech' patterns or verbal comments m any way which 
would te%d either to support sjyudfe^t activity or to beUt- 
tle students (3). 

UrireheptivelDisapproving—\i in the process ,of lectur- 
ing. ^dlscusslng, or questioning jh^ instructor makes 
verbal comments which tend to belittle students or show 
disapproval for students' comments, the instructor is re* 
jl corded as gnreceptive (2) )»r highly unreceptive (1). 

, ^ d) Non-Verbal Expression 

SupportivelReceptive-r-^i ih the process of lecturing,^ 
discussing, or questioning the instructor attempts to^' 
keep students on task or to reinforpe student behavior 
and comments by using changes in physical position or 
facial expressions, then het)r ,she is recorded as being 
receptive. For example, while lecturing about a ^raph 
on the chalk board the instructor walks toward t|te 
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, " board and points to the appropriate points on the 
graph, while students ar6 discussing a given problem 
the.in^tructor nods and smiles^in agreement, while ask- 
ing a question the instructor takes/a step toward the 
student'-Once again highly receptive (5) and receptive 
(4) are used to indicate degree of/supportrve instructor 
action ^ 

t^eutral — The instructor^does no/ demonstrate any phy- 
sical movement or facial expression which would tend 
to b*e supportive or discouragirVg to students (3). 
UnreceptfvelDfsapprovmg — If in ihe process of lectur- 
ing, discussing, or questioning the instructor uses plw- 
* Steal gestures?, movement^ or facial expressions which 

tend to demonstrate lack of concern for stud^s, or 
disapproval of a studenVs comment, the irfslructor is 
recorded as unreceptive (2) or highly unreceptive (1). 
The coding form is segmented vertically by 2{j^second time in- 
tervals sequentially labeled 0(/l, 002, 003, an^ so on, to 138. Upon 
receiving a 20-second beep, the reviewer records what. the GSI is 
teaching For instance, on me 34th beefM'11 mm. 20 sec. after the 
start of class or the videotape) assume the reviewer observes the 
GSt initiating questions liased on a real world problem which in- 
volves using oneBConornic principle to obtain a^sotutlon. In pos- 
ing the question the GSI implies ihat even the "dumbest' student 
in class sljould know the answer. Assume also that the reviewer 
observes the GSI tu/n and fate J& way from the*student in posing 
the question Usind^ the operational definitions given aboy.e, the'' 
reviewer would re^iord such an ^servation as follows. 
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. ^ fhe 3 under Method mdicates that the instructor initiated the 
.Question or problem The 4 for Objective indicates that the in- 
structor IS looking for a' simple application of theory. Under the 
heading Expressions the rwo 1 s reflect thelact that the instructor 
gave both nonverbal and verbal signs a(\ strong disapproval and 
unreceptiveness to the'Studeots.® ^ » . + 



•A Videotape demonstrating tK 
jectiV^s. and expressions has bedn 
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alternative teaching methods, ob- 
eveloped by members of the project 
he Center staff (Salemi and Becl^^ 1974). 



Tables 



Summary Checklist for Videotape Reviewing / 

I Summary of Videotape Review Coding Data ^ . 



Score 


Method 


5 




4 




— 3 — 




2 




1 





Learning Objo^cttve 



\f^rba\ Expression 



Nonverbal Expression 




II Exposition Skills and Physical Characteristics 
1 Eye Contact ^ 

2. Mannerisms 

3 Voice (pitch, rale, volume, articulation) 

4 Delivery inj^neral 

5 Use of biackbo^ftltneatnessy^se use of spac^ 

III Stupi^nt'lnstruchr Interaction 

1 Question technigj^s (open, closed, directed, reversed, re- 
layed, rhetoric^, thought-provoking, pause for answers, 
clear, preconceived) 

2 Attitude pfojected (positive, negative or indifferenV 



3 ^Motivation of the students 



J After reviewing the entire tape, the reviewer simply calculates 
the piercentage of time the GSI spent in using different methods,, 
objectives, and expressions ' ^ - 

Summary \ Checklist. Filling out the Summary Checklist for 
videotape reviewtng is a less formal and more subjective proced- 
iJre than coding, ^s shown m Table 8, the Summary Checklist is 

^ divided into four categories — t.e., sumpriary of videotape review 
coding data, exposition skills and physical characten^ticsr stu- 
dent-instructor interaction, and organization and objectives. As 
one goes down the checklist the categories become increasingly 
more subtle and difficult to cqrrect. If it is a>i instructor's first tap- 
ing^episode, the ret'iewer/crituquer and the'instructor shoald prob- 
ably concentrate on Category lr>^ the'third taping, they wi/l have 
dealt with Category II and III prODtems and they can concentrate 
on Category IV Tfie Summary iChe^ktTst actually has two pur- 
poses first. It provides important dataSQr preparing th<* critique, 
andjsecond, a copy of the checklist'can be given to the mstruGtOf 
for reference aftjer the critiquing session, . *' 

Preparing the critique. To prepare the criliquTng session the 
reviewer/critiC|uer must first schedule a time when the GSI is ^ree> 
for at least two hours. Then all (he'datrf must be collected and 
synthesized — le, coding forms. Summary Checklist, GSTs Daily 
Jeach^ng Checklist, student evaluations (if available), any data 
from previous tapmgs, and the tape .itself. ^ Using all this informa- 
tion the reviewer/critiquer gnust set objectives for the critique. 
These obyectives will depend on the instructor and how many 
times that instructor has been taped. A list should be madettf all 
the points to be made as'well-^s when to stop the tape. At this 
pbint the'critiquer IS prepared. . ' 



Table S (cont'd) 




IV Organization anjiJ^^ectives 

^1 Gep^efaTstructure of class \ 

^(Introduction, provide refereqceVor starting jioints, ^ 
Sub-parts summarized fnd referred to outline; <J 
^ Ending summarized and projected Jo the next session) 



2 Did the instructor meet the objectives he or she sef^ 



3 Are Ihe objectives of the class^^ssion consistent with the 
course goals? 
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Handling the critique » 

HoVa critique »s actually conducted depends on the reviewer/ 
critiquer and the mstructor. It is a very subjective process and no 
'fornlula exists for a successful critique. Following is a list of sug- 
gested principles which have proven helpful 'at Minnesota 

• The reviewer/critiquer should Jry to establish a friendly, re- 
laxed atmosphere. " " , 

• The reviewer/criti*quer should be non-judgmental. The pur- 
pose IS tO'improve instructors, not'judge them. 

The'review^r/critiquer's rote is to rriake suggestions to the 
instructor, which may be adopted or disregarded, 
' • The reviewer/onjiquer shQ^|jld not.'trylto charige an instruc- 
tor's style but instead work on improvi^i^ that i;istruetor*s tech- 
niques and skills.. t 

• The review^r/critiquer should* follow good communication 
. priVictples during the critique. It is quite unconvincing to ex- 
plain to an mstructor that eye C9ntact is. important while you- 
stare at the fceiling. * , ' j 

• The reviewer/critiquer should take personalities intd account. 
Some people are more defensive than others, and the criti- 

^ quer must be ready tcT'adjust remarks accordingly. 
* ' The reviewer/critiquer should provide the instructor with 
motivation. 

• The instructor ^^uld be encouraged to trV to s'ee himself or 
herself as the students do. Many valuable insi'ghts are'obtained'; 
this way. " ^ ^ 

• The reviewer/critiquer should make clear from the beginning 
. > -/what IS expected from the instructor^e?ore thafirst critiquing 

trte instructor must be. notified thdt'the Daily Checklist must be 
completed prior to the videotaping. , . . 

p. The reviewer/critiquer should make sul"e that there is an' 
agreement as to what the GSI will work on and make a note of 
. thi% for the next taping session. \ • 

• The revteyver/critKjuef should not try tp do too* much in one 
^ session, a critique of a one-hour class should-be kept to about 

two or twp-anchone-half hours. ' '1 ' 

Experience at Minaesota has indicated that at least three full 
.^^videotaping episodes are rtecessary for most of the ihexperh 
enced GSI's in the. program^ 

• The "revifewer/critiquer should attend the' GSI seminars. 
Teaching skills learned in the seo^inars ban be discussed and 

^ ^ reinfoVced during the critique, • ^ . , 

• The videotaping and feedback protess should be completely 
. confidential. The instructor should have complete control over 

who sees his or her tape ' . 

• 27 • - 




^alyating the 
Effectiveness ; . 
of theGSI 
Training Program 

This section reports on the results of an evaluation of the ef- 
fectivene^ of the GSJ tj^nlng program at the University of Min- 
nesota.* " - * 

Throughout the f|rsf year m the^evelopm^t of the GSI train- 
iijg program, 1970-71, both participation and feedback from the 
GSI's had been exceH6nf. However, a basic question remained. 
Were we having any measurable impact on student and instructor 
performance m the actual classroom? To resolve this question, the 
following s\udy was conducted. 

♦ 

Experimental Design ' . 



During the 1971 fall qliarter, aV students enrolled in Economics 
1-001 (Principles of Economics — Macroeconomics) were selected 
as a control population. Students in the course met once a week 
for a mass lecture, and were divided into fourteen sections yvhich 
met as sections three times each week. Average section size "was 
25. ^tudeht enrollment m each section was essentially a self- 
selection pMtess on a.fifSN;ome, firatsfer^^ ^asisngepHbns \^ere 

^"fered at various times between 8.00 a.m. and 4.00 p.m^.r MontJay 
through Friday, w'lth nq instructors identified^in class listings The 
mass lecture wa6 handled by senior faculty in the department 
^ while the fourteen, sections were conducted by seVfen graduate 
student instructors, each teaching ^wo sections. 

'Dgring the fall quarter, the seven GSI's of Economics 1-001 
i\ere precluded frgm participating m or having knowledge about 
the videotaping, seminars, or any other facet of the training ^ys- 

. tem described in this phonograph. Similarly, these seven instruc-^ 



* »The evxiluatjon design and results of the study reported in this j»ctiort 
have been adapted from an earlier^rticle published by D R Lewis and 
Q ;. Orvis (1973). ' ' " / 
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tors and their students w^re unaware of both the expe/iment.al 
design ancJ iht^ hyjoutheses being tested However all of the Eco- 
nomics 1-001 students m the fall term responded to survey qu^- 
tionnaires dealing with student characteristics and were pre- ^"d 
post-tested on the Test of Understanding in College Economics 
(Part I, Forms A and B) Post-course student evaluations of each 
instructor s performance were also collected on the Purdue Rating 
Scale fcJ^College Instructors 

In order to control for-lbe experimental training of instructors, 
the same severi GSl s w^re used as the experimental group during 
thW-wjnter quarter when Economics 1-001 was again offered. The 
experimental group of 438 winter quarter students was again di- 
vided into fourteen sections with an average section size of 31 As 
With the control students, all the Winter quarter experimental stu- 
dents Responded to the survey tjuestionnaire, the Purdue flating 
Scale and the TUCE Subsequent tests on selected-student char- 
acteristics and Pre-TUC£ scores revealed no significant differ-' 
ences between the conjrol- and expeijjnental groups (see Table 9 
and the study results below) All sections and instructors m both 
the fall and winter quarters used the same instructional materials 
^and departmental course syllabus, and, the same senior faculty 
gave the mass lectures ^ 
- The experiment was designed such that the seven instructors 
were randomly selected from a total of 22 GSTs »n the fall of 1971. 
The seven instructors were then given only a syllabus and sectiOn 
assignments and were not provided with any other assistance or 
training. Howeve^>cluring the winter quarter these same seven 
GSl s were system at ic^y^posed to the GSl training system de- 
scribed m this monograph ^^"^^ 




'^The Test of UnCJCTstanding *in College Economics (TUCE) js a na- 
tionally norrned and validated test designed to measure student perform- 
ance m the introductory economics course Based on recommendations of 
the American Economic Association s Committee on Economic Education, 
|t wa^ put)li^h€|d*by |{)Q^syj^ological Corporation (1^68) 

"Although most-ot^the GSl s in this experimental study had .teaching 
expeuence prior to their Rarticipa.tion in the prOjecl, a weakness inUhe 
d^^Ign of the study is the possibility that any superior performance in the 
wintOr quarter may be attributed^ to their .maturation and/or additional 
experience However, similar data on student and. GSl performanc^e from 
an oarlior study with the introductory course at the University of Minnesota 
indicated that the additional experience of only one term is not significant 
(Lewis and Dahl 1972) In fact, the obverse was true m the current study. 
The GSl who had the most previous teaching experience was the GSl "in 
the fall t^rm with the lowest student and instructor performance and sub- 
sequently ^showed the most improvement as a result of the training system. 
QiQ^the oth^r h.gn'd, the two GSl &w,with the least teaching experience were 
amongCthd top three in stude^U and instructor performance during the 
fall term, ♦ * 



Description of Experimental Results 



As Table 9 indicates, the winter quarter experimental group of 
students did not differ significantly from the fall quarter control 
group m any of five matching variables — i e . Sex, Age. Cumulative 
Grade Point Average. ACT Score, and Pre-TUCE-^at the two-tailed 
05 criterion lev^ emplo^d in the study Consequently, with the 
same instructors teachi/ng in both thsg fail and^winter quarters, the 
groups were considered adequately matched for purposes of the 
evaluation study ^ * . 

The Pre-TUCE data in-Table 9 also indicate that tb« Minnesota 
scores for both the experimental and control groups approximate 
yie aational norm of 13 24 at the Outset of each quarter term. 
,Post-TUCE scores for the fall quatfer control group also approx- 
imate the national norm of 19.08rfurther indicating normality for , 
the control sections (Psychological Corporation 1968). . ' 

^ V 

Impact of t^e Trainin^ystem . 

on Student t::ean^ina/ 

. . -^^ 

* As Table 9 indicates, the winter quarter experjmental students 
clearly outperformed the control students in economic unddr- . 
standing Not only were 4:he differences between grpup f?ost-TUCE 
scores' significant, but the Change-'in-TUGE scores (Post-TUCE 
minus Pre-TUCE) also indicated^significant differences The ex- ' ; 
pewmental group exhibited a 54 f^ercent gain over their Pre^UCE , 
score while the control grpup.experienced only a 43 percent gain 
m output added ^2 The gain^ for the controj group are comparable 
to the. national normmg data fox the TUCE wherein stucjents from ^ 27 
four-year colleges showed average gains^of 40.3 percent. The ex- . ^ ^ 
^ perimental grotjp's performance was clearly superior. 

Although the significance bf the traihing system'sNmpact /)n . 
student learning is clearly evidefit from the above data and pis- 
^ • cussion. a moro, controlled analysis of the data was p^fformecLby 

-•. ^ • ^ . J - 

' '^Any discussion of output added on the TUCE^wsi Jaemialified wilW ♦ 
the recognition that the output added function is clearly norvTTfroa^. there 
are easy (Questions, questions of medium difficulty, and some which are 
very difficult. In fact, the test was designed tbiyway in terms of cognitive 
composition It is therefore somewhat mappropriate to compare incre- 
ments on ihis test as constructed For example, at the extreme a stuctent 
moving his or her total score from 3 to 6 on the tOCE has picked up 
much less* economics than ^ student moving from 28 to 31 Only on a 
Q i iMear tQSt can these types of comparisons be safely made 
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Table 9 

^escHilpiion<)J Student ChawplgfisticC 
Performances, andEj^fuations: 
Fall and WipJ{»rl3uarters 

Fall^Qtrarler Winter Qlr. l-lest 

323 ^* N = 438 Comtmqng 

Variables ^"^"-^ — ^4.4^Seclions 14 Sections Means 

~ ~/ u ^ ' r 

Means^ D ^"^^-^^M^ans ^.D. 

Sex (0. 1)"" 71 45 ^>0--^6 30 
,Male=1 

Age(17KJ8)* ^ 20 49 3 09 l^J^O 2.74 "l 43 

Grade Point Av(^rage 2 73 5l 2 77 481 1.09 
(0-4) 

\ 

ACT Score (0-36) ^ X24.88 3 18 • 3 63 1.50 

•- ■ ^.Z'-^, 

Pre-TUCE (0-33) 1352^^ 3.70 13.04 Na 96 1.71 

post^TUQE (0'3b) ■ 1 9 4K 4 70 ^ 20 1 1 4.55v 1 9>! 

^ \ 

Change-in-TUCE , 5.94 4^50 \ 7,^7 457 \ 3,36- 

Awehqe Instructor Rating 4 vi 84 ^ 46* ^ 7^- 5^ 
(1-6). 1 = Very Low 

Ratm^Bcale Sub-parts 

1) Personal Evaluation 441 gp ^^4 76 ^5 ""^"-^ 5.42' 
(1-6), 1 = Very Lov« 

2) ^bjectiyity Evaluation 4 32 ' 91 / \4aO 86 ^5.82; * 
J(l-J6)."l =Very Low * ^ • ' ^ • ^ 

^^^t^Ro^itiOQ Ev<^atiqtL 3-32 1 IP . ^ 4 32 >99 '6.45-- . 

* ,^/i-a). j=yerKow • T;;^'^^?Nr^^ -x 

4) Testing Evaluation ' 389 ^ 95\ 410 ' .92\ V3.06" 
(1-6).'l =Very Low ^ *\ ' - ^ ^ \> 

5) Knowledge Evaluation 426 93' 4v52 81 ! 4.02" 
(1-6). 1 = Very Low \ • 

' Significa/it at the 05 lev.el X 
••Significant at the 01 le<rel 01 ' 





fitJtng the student descrtptors, evaluations, and iest results tp a 
rfti^ltiple linear regression model. 

When this was done, the significance of the earlier t-statistics 
was confirmed. ^3 Wh^le controlling for prior knowledge in eco- 
nomics, mental ability and achievement, maturation, sex, and stu- 
dent evaluations of the insti^tor, the type or class with experi- . 
mentaJ involvemeni in the project did have a significant associa- 
tion with the students' F^ost-TUCE scores.^* The model predicts that 
a student attending a class which was involved with the GSI train- 
mg system would, on the av.erage, score almost thre^-quarters oif 
one point 471) more than* non-participants on their Post-TUCE 
scores. 

. The regression model alap indicates that the six other variafbles 
< si^nilicaotly associate with student achievement in economic un- 
derstanding. PrioV knowledge jil* ecdnomlcs, mental ability and^ 
achievement, maturation, sex, and student evaluations of the in- 
structor were al^ found to be si^nifrcant. These findings are all 
.con^stent with the results of other research in this field (Bach and 
Saunters 1965, Bellico 1970, Capozza 1973, Gfery 1970, f!972; Lee, 
KetlBVi^nd Weisbrod 1970, Lewis and Dahl 19^2, Lewis and Orvis 
1971. plti^ and Moyer 1969, 1972, Saunders 1970, 1971, 1972; 
Sloane 1972^\Veidenaar 1972; Welsh 1972). 



Impact of the TrainingSystem 
on Instructor Performances . 

The data in Table 9* and Table 10 also confirm th^t the GSI 
traiAng. system had^measurable and significant influences on the 
instructors actual p*erformances as measured by studei^t evalua- 
tions. Not only was there a significant difference betweer) quar- 
"ne^sin the total Rating Scale for|all of the instructors, but each of, ^ 
thes&b-parts to the Rating Scale was' also significantly different 
between the experimental and control groups. In^turn, these^ 
changes in GSI performance associated significantly with student 
learning, as confirmed in the regression model by the association 
• of the instrCictor evaluation variable with Post-TUCE scores 

It IS imporfant to note that throughout the experimental quar- 
ter s s(ideotaping .review sessions the instructors were presented 

'3RegressiOQ coefficifehls and tests of significanpd can be found in 
Lewis and Orvis. op at , v. ' 

^ *" V * V 

i*AH variables i/i this^modeJ were, found tp *^ave^iotar-corremtions of 
.^21 or le^s in ^t\e correlaUon matrijl except ACT and Pre-TUC^ Ttiey tiarf 
a correlation of .31, a" degree of inter-corraleittoD ool ctetrimenfal '<o 
• model Analysis since they were both significantly cprrelal^d with Post- 
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with the Rating Scale (student, evaluation) results from their pre- 
vious course^ Suggestions and strategies for improvement were 
then developed with each instrugitor for each low-rated item. The 
instrumen^ancJ these procedures were apparently effective. 

■ f 

Individual instructor ratings on the Rating Scale and its sub- 
• parts are summarized in Table 10 for each of the two quarters. - 
With one major exception (Instructor IV), all of the GSI's increased 
their scores for almost all of the Rating Scale sub-parts. It is im- 
portant to note that the only irxstructor whpse ratings dropped (In- 
st|;uctor IV) developed mononucleosis during the expierimental 
quarter and was the least active and enthusiastic participant in the 
training system. This illness^ and behavior undoubtedly carried 
over into his teaching performance. In testing, for example, he 
Sirnply pulled old exams from his files. It is also interesting to note 
that Instructor V was an office mate with Instructor IV and used 
"^he same tasts as did Instructor IV. Consequently, bofh instructors 
went down m their student ratings deaimg with "tests and grades." , 
Both the students and the, Rating Scale instrument are apparently 
sensitive to such behavior and circumstances. 



The student evaluations, as revealed by the Rating Scale, were 
also substantiated in early videotape reviews d^uring the experi- 
mental quarter. Both the reviewer's observatiipns and the actual 
videotape coding prdtedure revealed the same strengths and ^ 
v\»eai^nesses as the student evaluations of GSI performances, tiigh 
instructor ratings on Personal Characteristics" and "Exposition" 
skills were supported by high coding frequencies on "Suppor- 
tive and Receptive ' cat^gorieis of verbal and nonverbal expres- 
sions, high instructor ratings on "Subject Mattisr Knowledge" were 
supported by high coding fre'quencies on teaching methods other 3l 
than "Lectu/e ' and on higher level learning objectives such as^ 
Complex Applications. > jThe consistencies between these two 
instruments, along with the actual videotaped observational were 
pei;suasive eviplence in getting the GSI s to change their teaching 
b^avior. 



In summary, this study has confirmed that a systematic teacher 
training fjrogcam involving^Graduate Student Instructors of intro- 
ductoryredononu^Ss^iJKj^rf int^rated sen^ of instructionarsem 
"^ri^rs, student^ ^valuaticjps, and yideotaped^^servatioVis can have 
a significant and ffTea$lirabTejmpact on both^dent^nd instruc-^ 
tor performance in the actual classroom, So^ifjcally, it wls^Jound 
that as a result of the ir-ammg system, [ajpiudwi performance, as 
measured. by the TUCE, ar\d (b) instructor J^atings, as measured by 
the Purdujfe Rating Scale for CcSMege Instructors, both increased 
O iificanily. It was also found that instructor ratings, as nx^asured, 
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by student evaltiations on tr^e Rating Scale, associate highly with 
stuflent performance's on the TUCE. 

The experimental efforts and the results of this study confirm 
tWb* other institutions and departments similar to the Department 
of Bopaomics at the University of Minnesota can and should 
undertake greater responsibilities for providing their graduate stu- 
'dent, Instructors with teacher training, * 



/ 



%. Suggestions 
fpr Implementing 
the GSI Program 



• In this final, section of the nipnograph, a number of ad<^ional 
suggestions are offered to any faculty, (or student) group inter- 
ested if> beginning such a GSI teacher tratoing program in their 
own department or schooj. A number of recomqiendations are 
made which are general 'm nature and not related lo*iti| teaching 
role which might be assigned to a GSI. However, suggestion^ are 
also offered as to how the training program could be ads^pted to ^ 
department which employs GSI s as section (discussion) leaders 
rather than as instructors responsible for a coufTse, 



Suggestions in General 



Ind.ependent of th6 exact nature of any GSI's teaching role, 
there is at least one Sine qua non for the successful implemenfa- ^ 
lion of a teacher training program. It is essential that thBspdnss^- 
• ing department commit itself to the program's success. 

Those attempting to introduce a GSI program should recognize 
that the sponsoring department will likely be motivated by two 
considerations. First, the department should consider the imple- 
mentation of any sueh trammg^rogram a^further commitment to 
quality undergraduate instruction. Second, the department' should 
consider the program a part of the graduate students' professional 
training. ^ ' ^ • 

During the formative stages of the program, the departm^t 
' chairman and those faculty menibers directly responsible for cur- ^ ^ 
riculum review, the graduate student program, and uncfergradua^e 
instruction should review the program s design and offer criticism 
and.su^eslions As the program, is implemer^ejj. Ihese depart- , 
mept members should jOin the prograin staff and GSI trainees for 
at least one session to voice their sup]lrort and expectations, and 
the department Si)urpose ip supportmg tji'e program. Finally, when 
. the traming^program has matured (p^r'haps after one or more 
.years) fhe departgxent should declare that participation in the GSI 
, )painirig pro^Taun is a conditiori for employment as a graduate stu^' 
Q It mstruptor ' ' . ^ 
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A second crucial element for successful implementation of the 
program is development of motivation and an appropriately posi- 
tive afttitude toward the training program on the part of the GSt's.' 
As a first step in developiojg^this attitude the program director 
shOfeild choose the seminar coordinator and reviewer/critiquer from 
among ihe graduate students respected by their peers as excellent 
teachers and economists Although most GSl s are^iiighly moti- 

. vated to participate actively in the training program by their own 
desire to become good teachers, we suggest thai the program 
director seek additional departmental support designed4a appeal 
to the GSj s developing sense of professionalism. At the University 
of Minnesota, we have received departmental approval for the GSI 
training program to be considered as an economics Graduate level 

' course which can be formaHy recorded on'the*<3SI s transcnp*l. In 
addition, the department has* suppor4e^ efforts to provide GSl s 
with their ' best videotapes for their, own professional use (e.g,, 
as a vivid source of information on teaching performance and ef- 
fectiveness which (he GSI' might decide to offer to a prospective 
employer). 

Because of the importance of the GSI s attitude to the sulicess 
of the training progr^, it is recommended that ll^e GSI program 
staff be particularly, careful to set a gobd example of teacliing 
'behavior in the sessions which they conduct. They should attempt 
to recognize the needs of the GSI s (e.g., nervousness before a 
first day in front of a class or the need tp understand university or 
departmental rules). They should state goals for the program as a 
whole and objectives for each session, evaluate the participants'* 
on the basis of these goals and objectives, and engage in periodic 
, evaluation of program effectiveness on the basis of these goals 
and objectives. They should demonstrate any good teaching tech- 
niques they recommend to the GSI s not only when thelechniques 
are topical but also fhrUughotrHl^e-prog ram. Staff members should 
also commit themselves to keeping to the announced agenda and 
schedule in ordj^r to .impress upon the GSI s the importance of the 
agenda items. \ * ♦ 

It IS !&lso important to note that the GSI training program should 
not teach *ar.particular style of instruction. Although we are con- 
vinced Jhat* gQ^)d instruction proceeds according to the learning 
process model described in this paper, we despot suggest a spe- 
cific set*^r'6ble?ttves, traits, or tefchnidjUBS *aS* beiri^^^cbfr-ect!* 
Rather, we are convinced that each GSI should be encouraged to 
devTelop his or her own strengths and recognize his or her own 
weaknesses. ' ^ . 

The Ipw cost of the program described irTttTlB monogr^ptfls an 
^dditiorfal reComrxiendatlon for its adoption. Most Of;the deveipp- 



mental costs have already- 'Been assumed in the creation of the in- 
-struments for evaluatioti and codification and the procedures for 
implementation The initial purchase costs for t"ReT^ttrttvely main- 
tehancd-free video equipmerit totals approximately $3,00^ At a 
larger university it might well be possible to rent video equipment 
from' trte university audio visual center or share equipment with 
another department— p g , at the University of Minnesota all of the 
necessary videotaping for eight to ten GSI's each year is sched- 
uled within one-and-one-half quarters. The only other direct de- 
partmental costs are the Dart^trme^seta HOG for tw ogup ior (gradu- 
ate student) faculty and released tinie for^orle senior faculty 



Adapting the GSI Training Program 
to an Alternative Setting ^ 



Any GSI training program rptJSt, of course, be designed to help 
the GSI function in the teactfmg role assigned by the^epartmeht 
Because at the University of f^innesota. the GSI's in econocnics 
serve as cours.e- rnstruotors responsible jor planning and imple- 
menlingjl^etr ovyn courses, the program described in this mono- 
graph 'has been planned to instruct participants in a hroad range 
of leachinjg skills. At many universities, however, GSI'swunction as 
section leadjsrs, coordinating discussion and problem4solving for 
a small group of students to whom a senior faculty pei(;sbr^ lectures 
in a large group format. These GSI's will find seminar sessions 
devoted to lecture skills, setting course goals, or planning exam- 
inations less relevant for their immediate use. In what follows, 
suggestions ate ma(de as to how one might adapt the GSI program 
to a department wFip/e GSI's serve only as section (discussion) 
leaders. 




The seminar portipn of the program ^affords a natural oppor- 
tunity for the seminar co'ordinator to bring together the senior fac- 
ulty responsible for lecturing and planning the course ana the GSI 
section leaders. Thus, the seminar coordinator should (mdeavQr 
to have the senior facDIty person discuss with the GSI's tl" e obk 
r the large lectures and (6r the "section meetings "h/this 
am &'^^^^Jf^d^!e(^c^ "in settirig 'obje*c^es for 
their sections, c^TT^Seiter coordupate their section ses§jK jDs^^wilh 
^the lecture sessions, and will havre input into decision-making for 
the course which should nniotiv^te Jheir own performance. 1 : should 
be obvious that the seminar coordinator will be^successful m this 
part of the sessions only with the full support of the senior faculty 
j"erson(s), especially since the senior faculty person must be able 
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to state his or her objectives and be wilhhg to accept some GSI 
input 

It iS strongly recommended (and obvious) that crucial skill/ for 
the GSI section leader to develop are the abilities to engage stu- 
-deats-4n-apply4ng-pF4n&^ples~pf6sar^ted 4n the leotur^ and to draw 



them out with questioning techniques Students seldom participate 
(except as active listeners) in a large lecture 'session. Conse- 
quently, students will receive supervised practice in applying the 
cpncepts they are learning ojily via discussion m small groups, 
problem sets, and exams. In each of these cases, it will more like- 
ly be the GSI than the senior faculty person who interacts with the 
^^stuplents. The seminar coordinator should devote at least 30 per 
^ejit of tl^e total seminar time to developing GSI's discussion- 
Jeading and questioning skills. As discussed e3rlier in this mono- 
graph, microleachihg sessions designed to reinforce probing 
techniques are particularly valuable teaching devices. 

In sprte of the focus on discussion techniques, the seminar 
coordinator should alsjJ demonstrate lecture techniques (particu- 
larly blackboard skilis).since the GSI should be able to make a 
well-organized presentation during a class session devoted to 
exam or problem set review. However, if there is a necessary 
choice bet>A/een a session in which someone* lecture^ on black- 
board technique and one in which the participants first learn some 
technique and then practice th^ technique in the group session, 
th(B latter-approach is recommended. 

Jf the GSI's will be sharing responsibilities for writing exam- 
ination^, the 3emmaiL,aQoxd inator should devote at least one se ^a^ 
sion to examination writing and evaluation skills. If possible, the 
seminar coordinator shoultl time the seminar sequ,ence so that 
participants can read and comment on one another's exam ques- 
tions before they are due to submit exam questions to the senior 
faculty person. ^ * , 

k > ■ 

Finally, it is as important for the GSf section leader as for the 
GSI lecturer to, receive feedback pri his or her classroom per-^ 
formance. The reviewer/critiauer should be sure that evaluatiOfll 
forms permit students to CQrriment on and answer specific ot/es- 
tions about their weekly section meeting^ The reviewer/cjifiquer 
flhould t^e patns, -however, tij emphasi2e^the^dev^l<i^)merttal ' 
rather than the judgmentafl juses of evaluation respfts. The re- 
viewer/critiquer should also arrange to videotape^e GSI dtirirtg 
section meetings. Because the GSI's role as soiefion leaderj is, dif- 
ferent, the reviewer/critiquer Will em|)hasi2e;emfer,ent things in the 
critique, in paVticular. how thp GSI Ifeads^lne class in- discussion. 
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.draws them out with Questioning kt^efiniqu6s. and, in general, 
supervises tfieir participation ' 



Appendix A: 

Seminar 
Syllabus 
Outline 




Fall, 1974 



Meeting One 



A. Introductioh. 



1. Minnesota's participation m the Amenc^an Economic 
Assoetatiorr/JornTCouncif of Economic Education proj- 

' ect. 

2. Motivatio njor the^roject and for the participatttm of 



ERIC 



teaching associates in our departnr^ent. 

3. How the teacher training program works, 

4. Establishing a seminar schedule. /. 

B. Objectives and goals. * 

.1, 8jy the conclusion of today's meeting, all participants 
should be able to demonstrate m disco^^sioa their will- 
ingness to participate actively in the prograrrt/ 
2. By the conclusion of today's meeting all participants 
should feel less anxiety m anticipating their first day m 
class. 

C A .model of edocatton — how the learning process^,is ac* 
complished. ^> <^ 

b Some data and queS|tions in anticipatiqp^^ firs^ay nb^dj 

1 . The ?i?S.^s.(*tWrpi^ ^ . S - 

2. YouT^Trg^ds^fS iristnlfclors. ' i - 

3. .•Some University resources and how they^Work. 

E. A consjcJeration of poten^^t future topic 

1. Discussion: "How jsfiould we use^^h^ seminar time to 
meet our needs^ / ^ 

2. Extrapolation: ;,4how can we gaugecthe>needs of our 
students?" / • * # A r 

4 X) X 

z 




- V ^ ' ■ ■ ' 

II. Meeting T^o ^* * , / ^ 

A. Discovering the needs of students. 

1 At what level should we teach the prmotples coiftVe? 

2 Assumpt'ipns aj^out the backgrounds of students, 

B. Setting course goals. ^ ♦ 
C Using behavioral ob)ectlves1or^ach goals. 

D How sdttmq objectives makes testing and evaluation 
easier , ^ ' . • ' 

Ifl. Meeting Three; ^ r t 

A" Further discussion on imp<ementing the objective-setting 
approach to class preparation. 

1. Integrating the audiovisual program and the seminars. 
' . .2. Datacollgption. 

'B. Some notfs on the importance of teaching skills, 
o Lecture skills., ^* y 

IV.'^eeting Four ^ ^ ^ ^ \ 

A. Discussion of our objective-settmg experiment and the 
use of the daily teaching Checklist. ' ^if^ 
; B. Lecturing skills. ^ • ^ -.-^ « ^ 

1. Discussion of guest lecturer's comments at meeting 
' ^ thi;ee. ^ • - r . ' . 

2 te?N4jng^and the ute of ad\fanced organizers. ^ 

C. The use pf audiovisuaHools in lecturing. * 



^38 Meeting Five 



A HOW to write, give, and interpret effective examinations 
P Discussion of questions and problems submitted by par- 
ticipants. V, * •\ ^ 
1. What sVioalti an mstfuct^tdo wherrhis class slows him 
up with trivial qt^estions?^ ^ ^* ' • ^ . 
2 .How should Awe gVade our stCidents? What, are ^pod 
*\' / ;/ ' ■ rules.for giving grades? ^ \ • " 

* »» ^ 3 Does each section instructor have an obligation to 
. * ^ grade alike'? , * ' \ 



■ * VI, Meeting Six . ' ' . . ' ^ 

' • V . A Continuation of our idea exchange — specific techniques, 

4 , / 'problems, situations, and questions relevant to our tfeach- 

O , • ^ ing experience thus fc^r. ' - . 
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Br What kinds of feedback do we as teachers get from our 
students? * * A * - 

1. What kinds of student feedback give us information 
. useful in making midcourse corrections in,our courses? ^ 
f2. The use of student evaluations of instructors. 

C. '.The art of stimulating student feedback by askirfg ques- ^ 
tions. X 

1. What IS* probing? How cap prgbing shift the emphasis 
from instructor talk to student talk? 

2. What are the kinds of probes? vC'hat is each Kind of 
probing good for?,.. 

3 When shoul'd^an instructor probe students? 

VIL Meeting ^ven • ^ 

A. Ar)aly2mg the feedback we get^from our students. ^ 

1. Test results. • * 

2. Videotape. * ■ ^ • 
o ' 3, Instructional evaluations. 

• 4. Feedback via class dialogue. 

B. A videotape example of probirtg. 

VIIL Meetmgfight . ^ 

One of the rlabst difficult things about adiscU$sion is starting 
It with, a CQmmon experience oc a corttroversy. Thus, all 
members of the group^ould bring to the serfiinar on^ "dis- 
\ ^ cussicui starter" they have used successfujjy or think they 

might use successfully In addition to beginning discussions/ * " '. 
we will talk about' , * • a.' 

A.. The'purpos'e of discussion ' -39 

• B.. How to get participation. ^ *. ' * 

f .C.'^he instructor's role in diSiCussi6n. , . . /' . 

■ ^ ' D How tb" keep drsd:usaio*n from degeqerat/ng Nntp argu- 
ments, " , ' * ' " ' " 



IX. Meeting Nine " 

Obtaining- participants' feedback on the seminar portion of 
the Teacher Jrainmg Program .and other unresolved issues. 
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Appendix B: 

Categorical 
References 

^ The following list, while not exhaustive, categorizes refere^icies 
which the authors have found to helpful The' most useful have 
been starred and itahcized, ' * ' 

Topics dealing with Instructional planriing and'design (Davies 
1973, Gagne and Bnggs 1974) and the constructioa ^nd use of 
instructional objectives (Ausubel 1960, Biehler 1-974. Blopm 1968. 
^Gronluffd 1970, Kelley 1973, Krathwohl 1964, Mager 1962. Nelson 

1970. 'P^iillips 1972, 1974, ThompsoR 1970, Travers 1973) have an' 
extensive and cumulative body of Itterature in.the field of Educa- 
tion. And, of C9urs$. the "general educati6n" literature of h^her 
education is very Sfgnific^nt to- the goals* and objectives of a'ny.^ 
introductory course (Axelrbd "l 969, /Barzun 1970, *^Bell 1966:^ 
Bruner 1963, Dressel 1968, 1969, Freedman 1967, Mayhew 1968; 
Postman and ,Wemgartn^^t: 1969, Sphwab 1969^ Thomas 1960. VVor- 
•mald 1964)V fylcwe recently, a growing body of economic education 
literature has examined ajid reported on the go^s, objectives and 
Cu.tcomes specific to the principles course .in economics (Attiyeh 
and Lumsden 1972, ^Bach 1965, 1966, 1967, Bellico 1970, Clayton, 
1964. Crowley and-Wiltoa 1974. Dawson and Bernstein 1967. Leam- ^ 
er 1950, 1965. Lewis*1970, 1973^ Lewis and Dahl 1i972, Moyef and 
Paden 1968, J97Q,.t>aderT and Moyer 1971, 1972. Saunders 1970, 

1971. Saunders and Bach 197(3? Siegfried and White 1973). Even 
the jroles .of. and effects up6n attitudes and s/alues have been ex- 
amined as outcomes of the principles course (Horton 1972. Jacob 

^1958, KaPSter\sson and 'Vecider 1974, ^LOker 7972, Mann and Fus- 
feld 1970: Slo^f^fe 1972: Thompson 1973). . • 

In addition to ^ number of-excellent sourcefl|(t(i the higher edu- 
c^tio.n*Utefatufe dealing witfi general tip's for teacher training (Al 
iep 1968. Beard 1970. Bhg'h 1972, Eble 1970, 1^71,1972, Flournoy ^ 
•1972. Hansen 1973s H*ghet-.1^54, Lee 1967, MIT 1974\ ^McKeachie ' 
' 1969, Miller- 1972, Morris l97d\Nowle$. 1968, Pace 1973; PetersoK 
1946, Travers 1973), other referer^es are directed to the specific 
skills»ar>d.u?ie of /ejcfur'es (BeafdN970; BJiqh'1972; Buckles and" 
McMahon 1971, *McKeachfe 7959), the. construction and use of 
ests and measurements m the classroom (Buckles, and Welsh 
^Etel ^962,'Fels 1967, 1970, FursM964. Ger^idi^^ Gronlund 
Isaac ^nd Michiieri971 .-iewis a/id,DahU^971 . Matten 



1972, Psychological Corporation 1968, Rothman and. Scott 1973, 
Sanders 1966, Wefsh and Pels 1969,. Whitney J 970, Wood 1961), 
discussion techniques and interpersonal communication skills 
^(Greeriberg 1969, Gregory 1972, Lyon 1971, * McKeachie 7969, 
Phillips 1966>e^er 1969), and the evaluation ahd use of teaching 
assistants as disaJssionfeaders and teachers (Chase T970, Lam-» 
phear and McConnell 197QHs^^n§ apuiprv^s 1973, Nowles 1968, 
Gates and Quandt 1970). 

A significant number of articles have recently appeared in the 
, economic education literature which gwe systematic review and 
evaluatiQn to, a host of other special techniques— *i.e., games^and* 
simulations, programmed learning, audiovisual, computer assisted 
^instruction, laboratories, etc — which have been employed in the 
principles course (Attiyeh and Lumsden 1965, Ault and Rutman 
1974. Bach 196^, Boorris 1974, Brown and Flnc^ 1973, Calkins 
1970, Danielsen a>id^ St^uffer 197^, Dawson 1974, Dubbia and 
Taveggia 1968. Erriery ^nd Enger 1972," "Gordon 19JS9, Pels and 
Starleaf 1963, Pusfeld and^Jui^ip 1966. Haley. 1967, Hansen, Kelley, 
and Weisbrod 1970. Havrilesky t971, Kelley 1968^, 197^, Kotrilsky 
1^72, Levin ^967, Lewis^aod Orvis 1971, Lloyd 1970, Lumsden 1967, 
1970. Mgjrizi 1972, McConnell 1 964^ 1968, Meinkoth 1971 , Muhker- 
jee andn,oughlin 1971, Nelson 1959, Paden 1969^ 1970, Porreca 
^1971; Ramsett, Johnson, and Adarf^s 1973. Scriven 1^65; .Soper 
1973; Staff 19^2, Jiemann, Paden, and Mclntyre 1966. Villar41969, 
Widenaar 1972, Welsh 1972, Wentworth-andlewis 1973) ' 

/ 
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